Optimal Distribution of Computational Power in Free Viewpoint Interpolation by Depth Hypothesis Density Adaptation in Plane Sweeping

In this paper, we present a system to redistribute computational power in plane sweeping. Plane sweeping allows the generation of novel viewpoints of a scene by testing different depth hypotheses across input cameras. Typical plane sweeping approaches incorporate a uniform depth plane distribution to investigate different depth hypotheses to generate a depth map. When the scene consists of a sparse number of objects, some depth hypotheses do not contain objects and can cause noise and wasted computational power. Therefore, we propose a method to adapt the plane distribution to increase the quality of the depth map around objects and to reduce computational power waste by reducing the number of planes in empty spaces in the scene. First, we generate the cumulative histogram of the depth map of the scene. This depth map can be obtained from the previous frame in a temporal sequence of images, or from a depth camera with lower resolution or quality. Next, we determine a new normalized depth for every depth plane by analyzing the cumulative histogram. Steep sections of the cumulative histogram will result in a dense local distribution of planes; a flat section will result in a sparse distribution. The results, performed on controlled and on real images, demonstrate the effectiveness of the method over a uniform distribution and the possibility of using a lower number of depth planes, and thus a more performant processing, for the same quality.